Improving the energy efficiency of autonomous underwater vehicles by learning to model disturbances

Energy efficiency is one of the main challenges for long-term autonomy of AUVs (Autonomous Underwater Vehicles). We propose a novel approach for improving the energy efficiency of AUV controllers based on the ability to learn which external disturbances can safely be ignored. The proposed learning approach uses adaptive oscillators that are able to learn online the frequency, amplitude and phase of zero-mean periodic external disturbances. Such disturbances occur naturally in open water due to waves, currents, and gravity, but also can be caused by the dynamics and hydrodynamics of the AUV itself. We formulate the theoretical basis of the approach, and demonstrate its abilities on a number of input signals. Further experimental evaluation is conducted using a dynamic model of the Girona 500 AUV in simulation on two important underwater scenarios: hovering and trajectory tracking. The proposed approach shows significant energy-saving capabilities while at the same time maintaining high controller gains. The approach is generic and applicable not only for AUV control, but also for other type of control where periodic disturbances exist and could be accounted for by the controller. © 2013 IEEE

Simultaneous discovery of multiple alternative optimal policies by reinforcement learning

Conventional reinforcement learning algorithms for direct policy search are limited to finding only a single optimal policy. This is caused by their local-search nature, which allows them to converge only to a single local optimum in policy space, and makes them heavily dependent on the policy initialization. In this paper, we propose a novel reinforcement learning algorithm for direct policy search, which is capable of simultaneously finding multiple alternative optimal policies. The algorithm is based on particle filtering and performs global search in policy space, therefore eliminating the dependency on the policy initialization, and having the ability to find the globally optimal policy. We validate the approach on one-and two-dimensional problems with multiple optima, and compare its performance to a global random sampling method, and a state-of-the-art Expectation-Maximization based reinforcement learning algorithm. © 2012 IEEE

Towards improved AUV control through learning of periodic signals

Designing a high-performance controller for an Autonomous Underwater Vehicle (AUV) is a challenging task. There are often numerous requirements, sometimes contradicting, such as speed, precision, robustness, and energy-efficiency. In this paper, we propose a theoretical concept for improving the performance of AUV controllers based on the ability to learn periodic signals. The proposed learning approach is based on adaptive oscillators that are able to learn online the frequency, amplitude and phase of zero-mean periodic signals. Such signals occur naturally in open water due to waves, currents, and gravity, but can also be caused by the dynamics and hydrodynamics of the AUV itself. We formulate the theoretical basis of the approach, and demonstrate its abilities on synthetic input signals. Further evaluation is conducted in simulation with a dynamic model of the Girona 500 AUV on a hovering task

Direct policy search reinforcement learning based on particle filtering

We reveal a link between particle filtering methods and direct policy search reinforcement learning, and propose a novel reinforcement learning algorithm, based heavily on ideas borrowed from particle filters. A major advantage of the proposed algorithm is its ability to perform global search in policy space and thus find the globally optimal policy. We validate the approach on one- and two-dimensional problems with multiple optima, and compare its performance to a global random sampling method, and a state-of-the-art ExpectationMaximization based reinforcement learning algorithm

A Linearly Constrained Nonparametric Framework for Imitation Learning

In recent years, a myriad of advanced results have been reported in the community of imitation learning, ranging from parametric to non-parametric, probabilistic to non-probabilistic and Bayesian to frequentist approaches. Meanwhile, ample applications (e.g., grasping tasks and humanrobot collaborations) further show the applicability of imitation learning in a wide range of domains. While numerous literature is dedicated to the learning of human skills in unconstrained environments, the problem of learning constrained motor skills, however, has not received equal attention. In fact, constrained skills exist widely in robotic systems. For instance, when a robot is demanded to write letters on a board, its end-effector trajectory must comply with the plane constraint from the board. In this paper, we propose linearly constrained kernelized movement primitives (LC-KMP) to tackle the problem of imitation learning with linear constraints. Specifically, we propose to exploit the probabilistic properties of multiple demonstrations, and subsequently incorporate them into a linearly constrained optimization problem, which finally leads to a non-parametric solution. In addition, a connection between our framework and the classical model predictive control is provided. Several examples including simulated writing and locomotion tasks are presented to show the effectiveness of our framework

Lower body design of the ‘iCub’ a human-baby like crawling robot

The development of robotic cognition and a greater understanding of human cognition form two of the current greatest challenges of science. Within the RobotCub project the goal is the development of an embodied robotic child (iCub) with the physical and ultimately cognitive abilities of a 2frac12 year old human baby. The ultimate goal of this project is to provide the cognition research community with an open human like platform for understanding of cognitive systems through the study of cognitive development. In this paper the design of the mechanisms adopted for lower body and particularly for the leg and the waist are outlined. This is accompanied by discussion on the actuator group realisation in order to meet the torque requirements while achieving the dimensional and weight specifications. Estimated performance measures of the iCub are presented

Robot-object contact perception using symbolic temporal pattern learning

This paper investigates application of machine learning to the problem of contact perception between a robots gripper and an object. The input data comprises a multidimensional time-series produced by a force/torque sensor at the robots wrist, the robots proprioceptive information, namely, the position of the end-effector, as well as the robots control command. These data are used to train a hidden Markov model (HMM) classifier. The output of the classifier is a prediction of the contact state, which includes no contact, a contact aligned with the central axis of the valve, and an edge contact. To distinguish between contact states, the robot performs exploratory behaviors that produce distinct patterns in the time-series data. The patterns are discovered by first analyzing the data using a probabilistic clustering algorithm that transforms the multidimensional data into a one-dimensional sequence of symbols. The symbols produced by the clustering algorithm are used to train the HMM classifier. We examined two exploratory behaviors: a rotation around the x-axis, and a rotation around the y-axis of the gripper. We show that using these two exploratory behaviors we can successfully predict a contact state with an accuracy of 88 ± 5 % and 81 ± 10 %, respectively

Kinematic-free position control of a 2-DOF planar robot arm

This paper challenges the well-established as- sumption in robotics that in order to control a robot it is necessary to know its kinematic information, that is, the arrangement of links and joints, the link dimensions and the joint positions. We propose a kinematic-free robot control concept that does not require any prior kinematic knowledge. The concept is based on our hypothesis that it is possible to control a robot without explicitly measuring its joint angles, by measuring instead the effects of the actuation on its end-effector. We implement a proof-of-concept encoderless robot con- troller and apply it for the position control of a physical 2- DOF planar robot arm. The prototype controller is able to successfully control the robot to reach a reference position, as well as to track a continuous reference trajectory. Notably, we demonstrate how this novel controller can cope with something that traditional control approaches fail to do: adapt to drastic kinematic changes such as 100% elongation of a link, 35-degree angular offset of a joint, and even a complete overhaul of the kinematics involving the addition of new joints and links

Towards Minimal Intervention Control with Competing Constraints

As many imitation learning algorithms focus on pure trajectory generation in either Cartesian space or joint space, the problem of considering competing trajectory constraints from both spaces still presents several challenges. In particular, when perturbations are applied to the robot, the underlying controller should take into account the importance of each space for the task execution, and compute the control effort accordingly. However, no such controller formulation exists. In this paper, we provide a minimal intervention control strategy that simultaneously addresses the problems of optimal control and competing constraints between Cartesian and joint spaces. In light of the inconsistency between Cartesian and joint constraints, we exploit the robot null space from an information-theory perspective so as to reduce the corresponding conflict. An optimal solution to the aforementioned controller is derived and furthermore a connection to the classical finite horizon linear quadratic regulator (LQR) is provided. Finally, a writing task in a simulated robot verifies the effectiveness of our approach